The current technology in commercial products such as pen-based graphical user interfaces (GUIs) is often derived from mouse-based interfaces. One goal of such pen-based systems is to mimic or at least take into consideration the mechanics of handwriting: fast, brief, and repetitive phasic muscle contractions that guide a process of rhythmically making small looping motions with a stylus or pen. However, pen-based GUI techniques in the prior art suffer from limitations and drawbacks that pose serious hurdles for users (e.g., the difficulty of learning a gesture set).
In order to facilitate the proliferation of such device interfaces, several factors need to be considered. Pen interfaces should seem fast and efficient to the user. Many conventional pen interfaces and techniques currently still build on familiar toolbars and pull-down menus, but these may require a round trip between the content area and the interaction area. Furthermore, because pens lack the state transitions offered by mouse interfaces, techniques often become burdened with timeouts or delays (e.g., tap-and-hold to bring up a menu) that the user must wait for every single time the corresponding action is invoked. Such delays are not only irritating to the user as proficiency increases with the device, but also prone to false activation, such as when the user unintentionally pauses with the pen during mental preparation (e.g., while thinking about what to do next).
Pen-based interfaces should be unambiguous. A pervasive theme in pen-based interfaces is the need for an Ink mode for leaving ink trails not subject to computer interpretation (intended for human interpretation), and a Gesture mode for indicating commands via pen input (intended for computer interpretation). Nearly all pen interaction papers in the prior art require, infer, or make a distinction between pen input that is Ink mode and pen input that is Gesture mode. Some systems infer the desired mode in some situations, such as combining inking and selection without prior selection of the mode. While this technique may be helpful for selection in some situations, the decision tree can get complex and only benefits the interface design with a single inferred mode (selection vs. ink). The approach is difficult to scale up to a system where the user may need a multiplicity of commands and gestural modes to use an application. With the latter approach it also may not be clear to the user what the interface will provide for each given stroke or input.
Sometimes special pen motions or gestures are used to switch to a temporary mode, such as dwelling with the pen (tap-and-hold or move-then-hold), making a dot (tap) with the pen after drawing an ink stroke to have it recognized, or interpreting a scrubbing gesture (scribbling back and forth to scratch out a previously drawn object) as a deletion command. The success of such mechanisms often depends on the user task, the specific application domain, as well as the details of exactly how an individual application interprets various pen events (tap, tap-and-hold, etc.). There also remains the possibility of false positive recognition (e.g., shading part of a drawing can be recognized as a scrubbing gesture, thus deleting it), since no gesture can be considered completely immune to potentially occurring in the course of freeform ink input.
The pen-based interface should be expressive. A fundamental problem with gestural interfaces in the prior art is that there is a delicate tradeoff in designing gesture sets that are learnable, memorable, and with gestures that are highly distinct from one another for the purposes of recognition. A few gestures can be recognized reliably, and recalled reliably by the user, but it becomes increasingly difficult to add more gestures and operations to an existing gesture set.
Selection-action command phrases represent a common pattern in user interfaces. The user first indicates a scope by selecting one or more objects, and then chooses a command that acts upon that scope. On the typical computer operating system desktop, the user can click and drag to sweep out a selection region, and then click on a tool palette or menu to choose a command. This typically requires a round trip between the work area and the tool palette or menu bar. Other means can be provided whereby round trips can be avoided by using right-click (e.g., on a mouse) to activate contextual menus or by learning and using keyboard shortcuts. However, this approach is less satisfactory for pens, as oftentimes no keyboard is available for shortcuts, and a button for right-click may be absent or awkward to use, making round trips unavoidable.
Accordingly, there is a substantial unmet need in the art for an improved pen-based user interface. Moreover, there is a further unmet need to provide rapid, unambiguous, and effective techniques for users of pen-operated devices to fluidly articulate selection-action command phrases.